Starting with today's restic master branch (restic/restic@24a2e5c), I updated go.mod to include the line:

replace github.com/pkg/xattr => github.com/greatroar/xattr solaris-fifo

After building restic with the updated xattr package, restic is able to successfully backup a directory containing a FIFO file:

mkdir repo target
mkfifo target/myfifo
restic -r repo init
restic -r repo backup target

That said... as far as I can tell, neither restic 0.14.0 release nor this updated build are actually backing up xattrs on regular files, but since the behavior is the same in the release version I'm guessing that has nothing to do with this change. (Although I see there is still an open issue in restic about not backing up ACLs on SmartOS, for example, so this appears to be a known issue.)

Most of the changes aren't directly relevant to this issue, only the O_NONBLOCK is necessary. Reverting back many of the changes from #62 concerns me due to the file descriptor lifetime issues addressed there.

Due to errors with doors and whatnot, my local copy of xattr currently has a function called to check if a file can have xattrs and is called from the various entry points into xattr:

func xattr_supported(path string) (bool, error) {
        var st = unix.Stat_t{}
        sterr := unix.Lstat(path, &st)
        if sterr != nil {
                return false, sterr
        }
        if ((st.Mode & unix.S_IFMT) != unix.S_IFREG) &&
            ((st.Mode & unix.S_IFMT) != unix.S_IFDIR) {
                return false, nil
        }
        return true, nil
}

Which isn't pretty and adds an extra system call for every path but avoids problems backing up parts of the root filesystem without having to add files to the exclude list. There are still filesystems like tmpfs which don't support xattrs at all, but you get unix.EINVAL there which gets changed to unix.ENOTSUP for the consumers of xattr.

Oh, I wasn't aware that that's what the NewFile calls were for. I just wasn't sure whether I could actually pass O_NONBLOCK to os.OpenFile. Pushed a smaller patch that does that instead. Judging from how os.O_* is implemented on Solaris, this should work, though it's a bit of a hack to combine os.O_* and unix.O_* constants.

Thank you guys - great work!
Unfortunately, I do not have access to Solaris, right now. So, if any Solaris' user can confirm that this PR works/fixes the issue, I'll be happy to merge it

@greatroar I have the same hack in the copy I've been using but you're right--- it doesn't seem legal.

Pushed a version that opens with unix.Open but uses os.NewFile on the resulting fds.

This looks good to me. I will see about testing.

xattr.Remove hangs with your built-in tests, see patch below. Also, the tests hang when I remove the O_NONBLOCK from xattr_solaris.go (as one would expect). I tried adding timer = time.AfterFunc(...) before the test and timer.Stop() after to catch the hang.

diff --git a/xattr_solaris.go b/xattr_solaris.go
index 57afaee..ce8b98a 100644
--- a/xattr_solaris.go
+++ b/xattr_solaris.go
@@ -87,7 +87,7 @@ func fsetxattr(f *os.File, name string, data []byte, flags int) error {
 }
 
 func removexattr(path string, name string) error {
-       fd, err := unix.Open(path, unix.O_RDONLY|unix.O_XATTR, 0)
+       fd, err := unix.Open(path, unix.O_RDONLY|unix.O_XATTR|unix.O_NONBLOCK, 0)
        if err != nil {
                return err
        }

This ensures the test runs in a bounded amount of time:

diff --git a/xattr_test.go b/xattr_test.go
index 2088288..36a3124 100644
--- a/xattr_test.go
+++ b/xattr_test.go
@@ -12,6 +12,7 @@ import (
        "runtime"
        "syscall"
        "testing"
+       "time"
 
        "golang.org/x/sys/unix"
 )
@@ -280,11 +281,15 @@ func TestFIFO(t *testing.T) {
                t.Fatal(err)
        }
 
+       timer := time.AfterFunc(30*time.Second, func() { panic("TestFIFO hang") })
+
        // We only care about this not blocking.
        _ = Set(fifo, "foo", []byte("bar"))
        _, _ = Get(fifo, "foo")
        _, _ = List(fifo)
        _ = Remove(fifo, "foo")
+
+       timer.Stop()
 }

Ok, added O_NONBLOCK in removexattr, and O_CLOEXEC while I was at it. I'd rather leave timeouts to the go test command.

Did anyone verify and also can approve the PR (so, I can be sure it does not break anything and I can merge it)?
Thanks!